A Guideline for Open-Source Tools to Make Medical Imaging Data Ready for Artificial Intelligence Applications: A Society of Imaging Informatics in Medicine (SIIM) Survey.

In recent years, the role of Artificial Intelligence (AI) in medical imaging has become increasingly prominent, with the majority of AI applications approved by the FDA being in imaging and radiology in 2023. The surge in AI model development to tackle clinical challenges underscores the necessity for preparing high-quality medical imaging data. Proper data preparation is crucial as it fosters the creation of standardized and reproducible AI models while minimizing biases. Data curation transforms raw data into a valuable, organized, and dependable resource and is a fundamental process to the success of machine learning and analytical projects. Considering the plethora of available tools for data curation in different stages, it is crucial to stay informed about the most relevant tools within specific research areas. In the current work, we propose a descriptive outline for different steps of data curation while we furnish compilations of tools collected from a survey applied among members of the Society of Imaging Informatics (SIIM) for each of these stages. This collection has the potential to enhance the decision-making process for researchers as they select the most appropriate tool for their specific tasks.

Automatic ARDS surveillance with chest X-ray recognition using convolutional neural networks.

OBJECTIVE: This study aims to design, validate and assess the accuracy a deep learning model capable of differentiation Chest X-Rays between pneumonia, acute respiratory distress syndrome (ARDS) and normal lungs. MATERIALS AND METHODS: A diagnostic performance study was conducted using Chest X-Ray images from adult patients admitted to a medical intensive care unit between January 2003 and November 2014. X-ray images from 15,899 patients were assigned one of three prespecified categories: "ARDS", "Pneumonia", or "Normal". RESULTS: A two-step convolutional neural network (CNN) pipeline was developed and tested to distinguish between the three patterns with sensitivity ranging from 91.8% to 97.8% and specificity ranging from 96.6% to 98.8%. The CNN model was validated with a sensitivity of 96.3% and specificity of 96.6% using a previous dataset of patients with Acute Lung Injury (ALI)/ARDS. DISCUSSION: The results suggest that a deep learning model based on chest x-ray pattern recognition can be a useful tool in distinguishing patients with ARDS from patients with normal lungs, providing faster results than digital surveillance tools based on text reports. CONCLUSION: A CNN-based deep learning model showed clinically significant performance, providing potential for faster ARDS identification. Future research should prospectively evaluate these tools in a clinical setting.

Lessons Learned in Building Expertly Annotated Multi-Institution Datasets and Hosting the RSNA AI Challenges.

The Radiological Society of North America (RSNA) has held artificial intelligence competitions to tackle real-world medical imaging problems at least annually since 2017. This article examines the challenges and processes involved in organizing these competitions, with a specific emphasis on the creation and curation of high-quality datasets. The collection of diverse and representative medical imaging data involves dealing with issues of patient privacy and data security. Furthermore, ensuring quality and consistency in data, which includes expert labeling and accounting for various patient and imaging characteristics, necessitates substantial planning and resources. Overcoming these obstacles requires meticulous project management and adherence to strict timelines. The article also highlights the potential of crowdsourced annotation to progress medical imaging research. Through the RSNA competitions, an effective global engagement has been realized, resulting in innovative solutions to complex medical imaging problems, thus potentially transforming health care by enhancing diagnostic accuracy and patient outcomes. Keywords: Use of AI in Education, Artificial Intelligence © RSNA, 2024.

Checklist for Reproducibility of Deep Learning in Medical Imaging.

The application of deep learning (DL) in medicine introduces transformative tools with the potential to enhance prognosis, diagnosis, and treatment planning. However, ensuring transparent documentation is essential for researchers to enhance reproducibility and refine techniques. Our study addresses the unique challenges presented by DL in medical imaging by developing a comprehensive checklist using the Delphi method to enhance reproducibility and reliability in this dynamic field. We compiled a preliminary checklist based on a comprehensive review of existing checklists and relevant literature. A panel of 11 experts in medical imaging and DL assessed these items using Likert scales, with two survey rounds to refine responses and gauge consensus. We also employed the content validity ratio with a cutoff of 0.59 to determine item face and content validity. Round 1 included a 27-item questionnaire, with 12 items demonstrating high consensus for face and content validity that were then left out of round 2. Round 2 involved refining the checklist, resulting in an additional 17 items. In the last round, 3 items were deemed non-essential or infeasible, while 2 newly suggested items received unanimous agreement for inclusion, resulting in a final 26-item DL model reporting checklist derived from the Delphi process. The 26-item checklist facilitates the reproducible reporting of DL tools and enables scientists to replicate the study's results.

Deep-learning for automated detection of MSU deposits on DECT: evaluating impact on efficiency and reader confidence.

Introduction: Dual-energy CT (DECT) is a non-invasive way to determine the presence of monosodium urate (MSU) crystals in the workup of gout. Color-coding distinguishes MSU from calcium following material decomposition and post-processing. Manually identifying these foci (most commonly labeled green) is tedious, and an automated detection system could streamline the process. This study aims to evaluate the impact of a deep-learning (DL) algorithm developed for detecting green pixelations on DECT on reader time, accuracy, and confidence. Methods: We collected a sample of positive and negative DECTs, reviewed twice-once with and once without the DL tool-with a 2-week washout period. An attending musculoskeletal radiologist and a fellow separately reviewed the cases, simulating clinical workflow. Metrics such as time taken, confidence in diagnosis, and the tool's helpfulness were recorded and statistically analyzed. Results: We included thirty DECTs from different patients. The DL tool significantly reduced the reading time for the trainee radiologist (p = 0.02), but not for the attending radiologist (p = 0.15). Diagnostic confidence remained unchanged for both (p = 0.45). However, the DL model identified tiny MSU deposits that led to a change in diagnosis in two cases for the in-training radiologist and one case for the attending radiologist. In 3/3 of these cases, the diagnosis was correct when using DL. Conclusions: The implementation of the developed DL model slightly reduced reading time for our less experienced reader and led to improved diagnostic accuracy. There was no statistically significant difference in diagnostic confidence when studies were interpreted without and with the DL model.

Older Tissue Age Derived From Abdominal Computed Tomography Biomarkers of Muscle, Fat, and Bone Is Associated With Chronic Conditions and Higher Mortality.

OBJECTIVE: To determine whether body composition derived from medical imaging may be useful for assessing biologic age at the tissue level because people of the same chronologic age may vary with respect to their biologic age. METHODS: We identified an age- and sex-stratified cohort of 4900 persons with an abdominal computed tomography scan from January 1, 2010, to December 31, 2020, who were 20 to 89 years old and representative of the general population in Southeast Minnesota and West Central Wisconsin. We constructed a model for estimating tissue age that included 6 body composition biomarkers calculated from abdominal computed tomography using a previously validated deep learning model. RESULTS: Older tissue age associated with intermediate subcutaneous fat area, higher visceral fat area, lower muscle area, lower muscle density, higher bone area, and lower bone density. A tissue age older than chronologic age was associated with chronic conditions that result in reduced physical fitness (including chronic obstructive pulmonary disease, arthritis, cardiovascular disease, and behavioral disorders). Furthermore, a tissue age older than chronologic age was associated with an increased risk of death (hazard ratio, 1.56; 95% CI, 1.33 to 1.84) that was independent of demographic characteristics, county of residency, education, body mass index, and baseline chronic conditions. CONCLUSION: Imaging-based body composition measures may be useful in understanding the biologic processes underlying accelerated aging.

Abdominal Body Composition Reference Ranges and Association With Chronic Conditions in an Age- and Sex-Stratified Representative Sample of a Geographically Defined American Population.

BACKGROUND: Body composition can be accurately quantified from abdominal computed tomography (CT) exams and is a predictor for the development of aging-related conditions and for mortality. However, reference ranges for CT-derived body composition measures of obesity, sarcopenia, and bone loss have yet to be defined in the general population. METHODS: We identified a population-representative sample of 4 900 persons aged 20 to 89 years who underwent an abdominal CT exam from 2010 to 2020. The sample was constructed using propensity score matching an age and sex stratified sample of persons residing in the 27-county region of Southern Minnesota and Western Wisconsin. The matching included race, ethnicity, education level, region of residence, and the presence of 20 chronic conditions. We used a validated deep learning based algorithm to calculate subcutaneous adipose tissue area, visceral adipose tissue area, skeletal muscle area, skeletal muscle density, vertebral bone area, and vertebral bone density from a CT abdominal section. RESULTS: We report CT-based body composition reference ranges on 4 649 persons representative of our geographic region. Older age was associated with a decrease in skeletal muscle area and density, and an increase in visceral adiposity. All chronic conditions were associated with a statistically significant difference in at least one body composition biomarker. The presence of a chronic condition was generally associated with greater subcutaneous and visceral adiposity, and lower muscle density and vertebrae bone density. CONCLUSIONS: We report reference ranges for CT-based body composition biomarkers in a population-representative cohort of 4 649 persons by age, sex, body mass index, and chronic conditions.

Identifying Patients with CSF-Venous Fistula Using Brain MRI: A Deep Learning Approach.

BACKGROUND AND PURPOSE: Spontaneous intracranial hypotension is an increasingly recognized condition. Spontaneous intracranial hypotension is caused by a CSF leak, which is commonly related to a CSF-venous fistula. In patients with spontaneous intracranial hypotension, multiple intracranial abnormalities can be observed on brain MR imaging, including dural enhancement, "brain sag," and pituitary engorgement. This study seeks to create a deep learning model for the accurate diagnosis of CSF-venous fistulas via brain MR imaging. MATERIALS AND METHODS: A review of patients with clinically suspected spontaneous intracranial hypotension who underwent digital subtraction myelogram imaging preceded by brain MR imaging was performed. The patients were categorized as having a definite CSF-venous fistula, no fistula, or indeterminate findings on a digital subtraction myelogram. The data set was split into 5 folds at the patient level and stratified by label. A 5-fold cross-validation was then used to evaluate the reliability of the model. The predictive value of the model to identify patients with a CSF leak was assessed by using the area under the receiver operating characteristic curve for each validation fold. RESULTS: There were 129 patients were included in this study. The median age was 54 years, and 66 (51.2%) had a CSF-venous fistula. In discriminating between positive and negative cases for CSF-venous fistulas, the classifier demonstrated an average area under the receiver operating characteristic curve of 0.8668 with a standard deviation of 0.0254 across the folds. CONCLUSIONS: This study developed a deep learning model that can predict the presence of a spinal CSF-venous fistula based on brain MR imaging in patients with suspected spontaneous intracranial hypotension. However, further model refinement and external validation are necessary before clinical adoption. This research highlights the substantial potential of deep learning in diagnosing CSF-venous fistulas by using brain MR imaging.

Visualizing Clinical Data Retrieval and Curation in Multimodal Healthcare AI Research: A Technical Note on RIL-workflow.

Curating and integrating data from sources are bottlenecks to procuring robust training datasets for artificial intelligence (AI) models in healthcare. While numerous applications can process discrete types of clinical data, it is still time-consuming to integrate heterogenous data types. Therefore, there exists a need for more efficient retrieval and storage of curated patient data from dissimilar sources, such as biobanks, health records, and sensors. We describe a customizable, modular data retrieval application (RIL-workflow), which integrates clinical notes, images, and prescription data, and show its feasibility applied to research at our institution. It uses the workflow automation platform Camunda (Camunda Services GmbH, Berlin, Germany) to collect internal data from Fast Healthcare Interoperability Resources (FHIR) and Digital Imaging and Communications in Medicine (DICOM) sources. Using the web-based graphical user interface (GUI), the workflow runs tasks to completion according to visual representation, retrieving and storing results for patients meeting study inclusion criteria while segregating errors for human review. We showcase RIL-workflow with its library of ready-to-use modules, enabling researchers to specify human input or automation at fixed steps. We validated our workflow by demonstrating its capability to aggregate, curate, and handle errors related to data from multiple sources to generate a multimodal database for clinical AI research. Further, we solicited user feedback to highlight the pros and cons associated with RIL-workflow. The source code is available at github.com/magnooj/RIL-workflow.

Association of Kidney Cysts With Progressive CKD After Radical Nephrectomy.

RATIONALE & OBJECTIVE: Simple kidney cysts, which are common and usually considered of limited clinical relevance, are associated with older age and lower glomerular filtration rate (GFR), but little has been known of their association with progressive chronic kidney disease (CKD). STUDY DESIGN: Observational cohort study. SETTING & PARTICIPANTS: Patients with presurgical computed tomography or magnetic resonance imaging who underwent a radical nephrectomy for a tumor; we reviewed the retained kidney images to characterize parenchymal cysts at least 5mm in diameter according to size and location. EXPOSURE: Parenchymal cysts at least 5mm in diameter in the retained kidney. Cyst characteristics were correlated with microstructural findings on kidney histology. OUTCOME: Progressive CKD defined by dialysis, kidney transplantation, a sustained≥40% decline in eGFR for at least 3 months, or an eGFR<10mL/min/1.73m2 that was at least 5mL/min/1.73m2 below the postnephrectomy baseline for at least 3 months. ANALYTICAL APPROACH: Cox models assessed the risk of progressive CKD. Models adjusted for baseline age, sex, body mass index, hypertension, diabetes, eGFR, proteinuria, and tumor volume. Nonparametric Spearman's correlations were used to examine the association of the number and size of the cysts with clinical characteristics, kidney function, and kidney volumes. RESULTS: There were 1,195 patients with 50 progressive CKD events over a median 4.4 years of follow-up evaluation. On baseline imaging, 38% had at least 1 cyst, 34% had at least 1 cortical cyst, and 8.7% had at least 1 medullary cyst. A higher number of cysts was associated with progressive CKD and was modestly correlated with larger nephrons and more nephrosclerosis on kidney histology. The number of medullary cysts was more strongly associated with progressive CKD than the number of cortical cysts. LIMITATIONS: Patients who undergo a radical nephrectomy may differ from the general population. A radical nephrectomy may accelerate the risk of progressive CKD. Genetic testing was not performed. CONCLUSIONS: Cysts in the kidney, particularly the medulla, should be further examined as a potentially useful imaging biomarker of progressive CKD beyond the current clinical evaluation of kidney function and common CKD risk factors. PLAIN-LANGUAGE SUMMARY: Kidney cysts are common and often are considered of limited clinical relevance despite being associated with lower glomerular filtration rate. We studied a large cohort of patients who had a kidney removed due to a tumor to determine whether cysts in the retained kidney were associated with kidney health in the future. We found that more cysts in the kidney and, in particular, cysts in the deepest tissue of the kidney (the medulla) were associated with progressive kidney disease, including kidney failure where dialysis or a kidney transplantation is needed. Patients with cysts in the kidney medulla may benefit from closer monitoring.

FDA Review of Radiologic AI Algorithms: Process and Challenges.

A Food and Drug Administration (FDA)-cleared artificial intelligence (AI) algorithm misdiagnosed a finding as an intracranial hemorrhage in a patient, who was finally diagnosed with an ischemic stroke. This scenario highlights a notable failure mode of AI tools, emphasizing the importance of human-machine interaction. In this report, the authors summarize the review processes by the FDA for software as a medical device and the unique regulatory designs for radiologic AI/machine learning algorithms to ensure their safety in clinical practice. Then the challenges in maximizing the efficacy of these tools posed by their clinical implementation are discussed.

THA-Net: A Deep Learning Solution for Next-Generation Templating and Patient-specific Surgical Execution.

BACKGROUND: This study introduces THA-Net, a deep learning inpainting algorithm for simulating postoperative total hip arthroplasty (THA) radiographs from a single preoperative pelvis radiograph input, while being able to generate predictions either unconditionally (algorithm chooses implants) or conditionally (surgeon chooses implants). METHODS: The THA-Net is a deep learning algorithm which receives an input preoperative radiograph and subsequently replaces the target hip joint with THA implants to generate a synthetic yet realistic postoperative radiograph. We trained THA-Net on 356,305 pairs of radiographs from 14,357 patients from a single institution's total joint registry and evaluated the validity (quality of surgical execution) and realism (ability to differentiate real and synthetic radiographs) of its outputs against both human-based and software-based criteria. RESULTS: The surgical validity of synthetic postoperative radiographs was significantly higher than their real counterparts (mean difference: 0.8 to 1.1 points on 10-point Likert scale, P < .001), but they were not able to be differentiated in terms of realism in blinded expert review. Synthetic images showed excellent validity and realism when analyzed with already validated deep learning models. CONCLUSION: We developed a THA next-generation templating tool that can generate synthetic radiographs graded higher on ultimate surgical execution than real radiographs from training data. Further refinement of this tool may potentiate patient-specific surgical planning and enable technologies such as robotics, navigation, and augmented reality (an online demo of THA-Net is available at: https://demo.osail.ai/tha_net).

Anonymizing Radiographs Using an Object Detection Deep Learning Algorithm.

Radiographic markers contain protected health information that must be removed before public release. This work presents a deep learning algorithm that localizes radiographic markers and selectively removes them to enable de-identified data sharing. The authors annotated 2000 hip and pelvic radiographs to train an object detection computer vision model. Data were split into training, validation, and test sets at the patient level. Extracted markers were then characterized using an image processing algorithm, and potentially useful markers (eg, "L" and "R") without identifying information were retained. The model achieved an area under the precision-recall curve of 0.96 on the internal test set. The de-identification accuracy was 100% (400 of 400), with a de-identification false-positive rate of 1% (eight of 632) and a retention accuracy of 93% (359 of 386) for laterality markers. The algorithm was further validated on an external dataset of chest radiographs, achieving a de-identification accuracy of 96% (221 of 231). After fine-tuning the model on 20 images from the external dataset to investigate the potential for improvement, a 99.6% (230 of 231, P = .04) de-identification accuracy and decreased false-positive rate of 5% (26 of 512) were achieved. These results demonstrate the effectiveness of a two-pass approach in image de-identification. Keywords: Conventional Radiography, Skeletal-Axial, Thorax, Experimental Investigations, Supervised Learning, Transfer Learning, Convolutional Neural Network (CNN) Supplemental material is available for this article. © RSNA, 2023 See also the commentary by Chang and Li in this issue.

The Evolving Role of Artificial Intelligence in Gastrointestinal Histopathology: An Update.

Significant advances in artificial intelligence (AI) over the past decade potentially may lead to dramatic effects on clinical practice. Digitized histology represents an area ripe for AI implementation. We describe several current needs within the world of gastrointestinal histopathology, and outline, using currently studied models, how AI potentially can address them. We also highlight pitfalls as AI makes inroads into clinical practice.

Few-shot biomedical image segmentation using diffusion models: Beyond image generation.

BACKGROUND: Medical image analysis pipelines often involve segmentation, which requires a large amount of annotated training data, which is time-consuming and costly. To address this issue, we proposed leveraging generative models to achieve few-shot image segmentation. METHODS: We trained a denoising diffusion probabilistic model (DDPM) on 480,407 pelvis radiographs to generate 256 âœ• 256 px synthetic images. The DDPM was conditioned on demographic and radiologic characteristics and was rigorously validated by domain experts and objective image quality metrics (Frechet inception distance [FID] and inception score [IS]). For the next step, three landmarks (greater trochanter [GT], lesser trochanter [LT], and obturator foramen [OF]) were annotated on 45 real-patient radiographs; 25 for training and 20 for testing. To extract features, each image was passed through the pre-trained DDPM at three timesteps and for each pass, features from specific blocks were extracted. The features were concatenated with the real image to form an image with 4225 channels. The feature-set was broken into random patches, which were fed to a U-Net. Dice Similarity Coefficient (DSC) was used to compare the performance with a vanilla U-Net trained on radiographs. RESULTS: Expert accuracy was 57.5 % in determining real versus generated images, while the model reached an FID = 7.2 and IS = 210. The segmentation UNet trained on the 20 feature-sets achieved a DSC of 0.90, 0.84, and 0.61 for OF, GT, and LT segmentation, respectively, which was at least 0.30 points higher than the naively trained model. CONCLUSION: We demonstrated the applicability of DDPMs as feature extractors, facilitating medical image segmentation with few annotated samples.

THA-AID: Deep Learning Tool for Total Hip Arthroplasty Automatic Implant Detection With Uncertainty and Outlier Quantification.

BACKGROUND: Revision total hip arthroplasty (THA) requires preoperatively identifying in situ implants, a time-consuming and sometimes unachievable task. Although deep learning (DL) tools have been attempted to automate this process, existing approaches are limited by classifying few femoral and zero acetabular components, only classify on anterior-posterior (AP) radiographs, and do not report prediction uncertainty or flag outlier data. METHODS: This study introduces Total Hip Arhtroplasty Automated Implant Detector (THA-AID), a DL tool trained on 241,419 radiographs that identifies common designs of 20 femoral and 8 acetabular components from AP, lateral, or oblique views and reports prediction uncertainty using conformal prediction and outlier detection using a custom framework. We evaluated THA-AID using internal, external, and out-of-domain test sets and compared its performance with human experts. RESULTS: THA-AID achieved internal test set accuracies of 98.9% for both femoral and acetabular components with no significant differences based on radiographic view. The femoral classifier also achieved 97.0% accuracy on the external test set. Adding conformal prediction increased true label prediction by 0.1% for acetabular and 0.7 to 0.9% for femoral components. More than 99% of out-of-domain and >89% of in-domain outlier data were correctly identified by THA-AID. CONCLUSIONS: The THA-AID is an automated tool for implant identification from radiographs with exceptional performance on internal and external test sets and no decrement in performance based on radiographic view. Importantly, this is the first study in orthopedics to our knowledge including uncertainty quantification and outlier detection of a DL model.

An Overview of Machine Learning in Orthopedic Surgery: An Educational Paper.

The growth of artificial intelligence combined with the collection and storage of large amounts of data in the electronic medical record collection has created an opportunity for orthopedic research and translation into the clinical environment. Machine learning (ML) is a type of artificial intelligence tool well suited for processing the large amount of available data. Specific areas of ML frequently used by orthopedic surgeons performing total joint arthroplasty include tabular data analysis (spreadsheets), medical imaging processing, and natural language processing (extracting concepts from text). Previous studies have discussed models able to identify fractures in radiographs, identify implant type in radiographs, and determine the stage of osteoarthritis based on walking analysis. Despite the growing popularity of ML, there are limitations including its reliance on "good" data, potential for overfitting, long life cycle for creation, and ability to only perform one narrow task. This educational article will further discuss a general overview of ML, discussing these challenges and including examples of successfully published models.